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Lessons are derived from research conducted at SERVE, the federally funded 
research and development laboratory serving the southeastern states. The 
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interviews with key informants from 15 school districts. The interconnected 
system of strategies derived from these sources includes those that: (1) 

build teacher capacity (professional development, teacher evaluation, and 
instructional leadership) ; (2) set the conditions in the district for 
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support effective school improvement processes (informal school reviews and 
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data on classroom environments, and defining grade- level proficiency) . Three 
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Introduction 



Purpose 

The primary purpose of this paper is to describe lessons learned about the district role in building 
teachers’ capacity to assess students more effectively, given that the lack of “assessment 
literacy” among educators has been identified by some as a key barrier to improving student 
achievement (Stiggins, 1999). A second purpose is to present a framework, based on 
experiences with districts, that will demonstrate how districts leverage an interconnected system 
of strategies in order to impact the quality of instruction and assessment in classrooms. The 
interconnected system of strategies includes those that: 1) build teacher capacity (e.g., 
professional development, teacher evaluation, instructional leadership); 2) set the conditions in 
the district for continuous improvement (e.g., defining standards, strategic planning); 3) support 
effective school improvement processes (e.g., informal school reviews, allocation of funds); 4) 
monitor progress (e.g., use of assessments that supplement state tests, use of data on quality of 
classroom learning environments, defining grade-level proficiency). 



Background of Project 

Lessons learned described in this paper are summarized from the first two authors’ work at 
SERVE, the federally funded Research and Development laboratory serving the southeastern 
states. Since 1990, SERVE has been supporting educators in the southeast by developing 
professional development resources (SERVE, 1998 and 2001) to help teachers improve their 
classroom assessment practices. The state policy context in the southeastern states has changed 
during that time span. 

Our experience of the 1990s in working with educators in the southeast can be conceptualized in 
terms of three approaches to assessment reform. The three approaches are briefly discussed 
below: 

1) The Alternative Assessment Movement: 

The assessment reform language of the early 1990s was alternative, authentic or 
performance assessment. The logic behind this movement was that how students are 
assessed makes a difference in how they develop as learners. Global competition and 
other forces were leading business leaders to conclude that the workplace of the future 
would require that “all American high school students must develop a new set of 
competencies and foundation skills if they are to enjoy a productive and satisfying life” 
(SCANS, 1991,p.vi). National discipline organizations developed content standards 
reflecting the emerging thinking about what. students needed to know and be able to do. 
In some cases, notably in mathematics, these national disciplinary standards found their 
way into state standards documents. 

The implication of these higher expectations for students was that assessment at the 
classroom, school, district, and state level needed to be more performance-based, 
reflecting these more complex learning goals. States and districts were interested in 
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using performance assessment to drive curriculum change in classrooms around these 
higher expectations. However, over time, states began to back away from performance 
assessments as the central part of state testing programs due to higher costs, difficulty in 
scoring and administering, and unanswered technical problems (Koretz, 1996; Asp, 
2000 ). 



2) The Standards-Based Reform Movement: 

A major influence on state policy that emerged in 1994 was the reauthorization 
amendments for Title I, which reflected a theory of how to drive progress toward higher 
standards for all students. This law envisioned that by the year 2000 all states would 
have in place content standards, assessments aligned with the standards, and mechanisms 
forjudging school progress toward achievement goals. Many states spent a great deal of 
effort on developing their standards for what students should know and be able to do 
during this period. Developing state assessments aligned with state standards was also a 
focus. The term standards-based reform implies that the task of laying out clearly what 
all students should know and be able to do is the first domino that, in turn, should cause 
all other reform dominoes to fall, including classroom assessment. 

3) The Accountability Movement: 

In the last years of the 1990s, accountability legislation - legislation that described how 
schools would be “graded” and the consequences of certain grades- proved a magnet for 
educators’ attention. The language of accountability reform was centered less around 
curriculum reform (and using assessment to drive more challenging teaching in the 
classroom) than on getting all students to grade level (as determined by a state test). 

Rather than supplementing the early 1990s goals of classroom change and curriculum 
reform, the accountability movement seemed to supplant the earlier goal, and focus some 
schools’ attention exclusively on getting more students to “pass the state test”. 

Whereas the policy “reform context” in the early 1990s (states experimenting with performance 
assessment) supported teachers’ interest in the use of classroom assessment methods and 
purposes that reflected needed “real world” competencies, the accountability context in 2002 
supports teachers’ interest in the use of classroom assessment that mimics competencies needed 
to do well on state tests. The implication of this change is that teachers are not likely to reflect 
on the quality or effectiveness of their classroom assessment practices. We believe some kind of 
structured support and intervention by districts is needed to help teachers understand the positive 
role classroom assessment can play in improved student learning. 

SERVE began work in 1995 with a small number of districts interested in building their 
teachers’ capacity for classroom assessment. Using assessment to improve student learning has 
consistently been identified as a weak area for many teachers (Crooks, 1988; Black & Wiliam, 
1998). Stiggins has argued throughout the 1990s for more attention to assessment in classrooms. 

“We have centered so heavily on the development of ever-more-sophisticated 
psychometrics and test development tactics for our high stakes tests that we have almost 
completely ignored the other 99.9% of the assessments that happen in a student’s life. 
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These are the assessments developed and used by their teachers in the classroom. If we 
seek excellence in education, then the time has come to invest whatever it takes to assure 
that every teacher is gathering dependable information about student learning, day-to- 
day and week-to-week, not just year-to-year. 

This action must be central to all future school improvement efforts, because if 
assessment is not working effectively day to day in the classroom, then assessment at all 
other levels (district, state, national or international) represents a complete waste of time 
and money” (Stiggins, 1999). 

The district plays a critical role in structuring the professional growth opportunities teachers need 
to translate the state demands for higher student performance into effective classroom strategies. 
Effective capacity building at the district level should create a “critical mass” of teachers and 
administrators who share a common assessment language and who understand how assessment 
can be used to help all students improve. Some evidences of teachers’ skillful use of assessment 
include 

S Use of assessment methods appropriately matched to learning goals 
S Use of exemplars or high level student work samples to help clarify teacher expectations 
S Use of rubrics to make expectations for particular assignments or skills clear 
S Use of formative feedback to help students improve the quality of their work 
S Use of data on learning to shape subsequent instruction and make decisions 
S Use of assessment information to motivate students by involving them as full partners in 
the assessment, record-keeping and communication process 
•S Use of assessment information to communicate effectively (e.g., using report cards, 
portfolios, and student-led conferences) 

Improving assessment practices as outlined above is difficult for teachers. Similarly, mounting 
an effort to build assessment capacity (change these practices in a critical mass of teachers) is 
difficult for districts. But some districts have set this goal for themselves as a necessary step 
toward raising standards and are learning some valuable lessons. This paper explores the district 
role in developing a critical mass of teachers who can use assessment to help students achieve at 
the higher levels envisioned by state standards and assessments. Although the focus here is on 
how to best support teacher change in the use of assessment methods and practices that develop 
students into better learners, the implications of the lessons learned are relevant to efforts to 
improve the quality of teaching, in general. 

Information Sources 



Realizing there is probably no optimum process for a district to build teacher assessment 
capacity, there are certainly some key lessons learned (procedural knowledge) to be gained from 
those who have worked to build teacher capacity in assessment. The lessons learned represent 
the conclusions of the senior author, Nancy McMunn, based on data in working closely with 
districts from 1995-2000 in providing an “awareness” level of professional development for 
teachers. In addition, the lessons learned build on collaborative work with Ken O’Connor who 
collected interview data from 15 districts as a way of throwing a broader net for generating 
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lessons learned. Both Nancy McMunn and Ken O’Connor also have participated since 1997 in 
the Association for Curriculum and Development’s (ASCD) Assessment Consortium. Site visits 
to ASCD Assessment Consortium member districts (who are considered to be “cutting edge” in 
terms of assessment reform) yielded notebooks of materials, articles, school and district 
observations, and other information around building assessment capacity. Districts visited were 
in Colorado, Vermont, Washington, Florida, and Arizona. Information gained from these visits 
was also considered in developing the lessons learned. The two main sources of information are 
described in more detail below. 

Source #1: Bay District Schools and SERVE Collaborate (1996 — 2000) 

SERVE has worked with Bay District Schools in Panama City, Florida as an Intensive Research 
and Development Site. Various instruments have been used with this district to collect pre and 
post assessment training data on teacher understanding, use of, and implementation of various 
assessment practices. Most instruments used (Appendix A and C) consisted of brief questions, 
with yes/no, Likert-scale, and open-response items which served to identify assessment issues or 
concepts the district needed to address in designing, developing, and implementing professional 
development activities. 

Bay District in Panama City, Florida began collaborating with SERVE in 1996 to implement a 
research and development project entitled Target/S ERVE. This district wide initiative involved 
training all 1,700 Bay District teachers and administrators in quality assessment using a resource 
developed by SERVE and modified for this district entitled: TARGET: Creating Effective 
Student Assessments. Unique aspects of this training were that, after the first two days of 
assessment training, teachers were required to develop and use a performance assessment with 
their students, which was then critiqued at a two-day follow-up session. During this same time 
period, the district mandated training of all teachers and administrators in curriculum alignment 
(training developed by the state of Florida around the roll-out of the newly developed Sunshine 
State Standards). 

This mandatory professional development on curriculum alignment and classroom assessment 
was one of the largest training initiatives ever in this district and it did provide a common 
message and language about how teachers should react to state standards. Survey data from over 
1,024 teachers showed that, after the assessment training, 80% reported needing additional 
assistance in the area of assessment. The responses most frequently reported were related to time 
for planning and practice to develop and use more quality assessments. Teachers also responded 
that they needed more help with the understanding, development, and use of rubrics. (See 
Appendix A). 

This district used the information collected from teachers to plan for the next level of work in 
building capacity. Some of the significant lessons learned were: 

A. The initial TARGET Training did motivate teachers to try different assessment methods 
or continue learning about assessment 

B. The overall use of rubrics was limited — teachers identified this area as a need for 
continued training and support. 

C. Very few teachers reported using rubrics to provide feedback - from conversations with 
some of the teachers it was clear that there was limited understanding of the use of 




Building Teacher Assessment Capacity Study 



5 



6 



SERVE 



diagnostic and formative assessment as ways to provide feedback to the learner and 
teacher 

D. The District needed to find ways to encourage schools and staff to find time for working 
collaboratively on classroom assessment 

E. It was difficult for teachers to view assessment as a process - it seemed that many 
teachers still saw assessment as an isolated event at the end of a teaching episode; thus, 
there was a need for the district to communicate a vision for “quality’ classroom 
assessment. 

F. There was some change in assessment methods used - however, the way the assessments 
were included in grades and the use of those grades were in question by the teachers and 
reviewing staff. 



In summary, the four-day training program, developed by SERVE and mandated for all teachers 
in Bay District did seem to have created an “awareness” among teachers about some of the issues 
in using classroom assessment well. The district has continued to require this training of all new 
teachers hired. However, it is not clear the extent to which “best practices” in classroom 
assessment have been implemented, particularly in light of the recent state push in Florida to 
grade schools, which seems to have resulted in teachers reverting to classroom assessment that 
mimics state test formats. 

Source #2: Interviews of Key Informants from Districts Involved in Building Teacher Capacity 
in Classroom Assessment (2000-2001) 

As a way of summarizing what has been learned about districts on the “cutting edge” of 
assessment reform, SERVE developed a structured interview protocol to use in the spring of 
2000 with key informants from 1 5 districts who had implemented some professional 
development in classroom assessment. Fifteen districts were selected for interviews based on 
their work with assessment and standards as evidenced by: a) invited membership in the ASCD 
Assessment Consortium, or, b) work with a regional educational laboratory, or c) 
recommendations from a consultant. 

The fifteen school districts were in six states (three in Mississippi, and two each in Alabama, 
Colorado, Florida, Georgia, and North Carolina) and one province of Canada (Ontario). The 
systems ranged in size from a district with 1,160 students to a district with 87,000 students. The 
percentage of minority students ranged from less than 2% to 95%. The fifteen districts were sent 
cover letters informing them of the interview protocol and inviting participation by a key 
informant who understood the system’s efforts in building teacher capacity. Both written and 
taped interview responses were collected. Ken O’Connor conducted the interviews for SERVE. 
The data were analyzed and themes summarized by question. These interview data were used as 
the second source for generating the lessons learned described below (See Appendix B). 

Lessons Learned 



It is abundantly clear from the data collected from the 15 districts and the work SERVE has done 
around building assessment capacity with Bay District (Appendix A) that this is a complex and 
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difficult task that takes multiple years to accomplish. As a way of reflecting on all of this 
information, the authors have summarized their impressions by generating a set of lessons 
learned that can inform future study of the district role. 



We describe fourteen lessons that we believe can help districts think about a process for building 
teachers’ assessment capacity. We have categorized the actions into four major areas that 
represent a district ‘thinking’ process for implementing new initiatives. These areas are Probing, 
Planning, Implementing, and Evaluating with Revising being a continuous part of the cycle. 




Building Teacher Capacity 

The Thinking Cycle 



Probing refers to consideration of the kinds of things that might be foundational for an effort to 
build teachers’ assessment capacity. For example, is there a strategic plan in the district that 
supports such an investment of resources? Time might be spent on articulating with others the 
need for the initiative and in brainstorming the questions or strategies that might define the 
initiative. 

Planning refers to laying out specific information around the how and what of the initiative. 
Figuring out the short term and long-term goals, designing an implementation plan, determining 
the targeted audience(s), determining exactly what the research and evaluation needs are, finding 
funding and/or support, obtaining buy-in from key players or coordinating with other initiatives 
or staff within the district and/or schools are all relevant here. 

Implementing refers to actually ‘doing the work’ to begin, support, and build capacity to sustain 
the initiative that has been outlined in the planning stage. 

Evaluating refers to collecting data and information to decide if the planning and 
implementation were successful and at what level. It helps to determine the strengths and 
weaknesses of what was done and guides future plans. 
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The organization of the lessons learned is not exhaustive or prioritized. The items described 
below are those that emerged from the information sources we reviewed. It is important to 
recognize that each item can stand alone, but each is also inextricably linked with others. The list 
that follows is not intended as a recipe but as a way of organizing what we gleaned from districts 
that were trying to build teachers’ assessment capacity. 



Probing 

1 . Have a vision or plan that supports standards-based assessment 

2. Take on the challenge of change in secondary schools 

3. Define standards-based assessment, especially as it relates to large-scale (state) 
assessments 



Planning 

4. Involve district leadership 

5. Create time 

6. Include school leaders (early) 

7. Ensure adequate financial resources 

8. Organize resources, especially staff, to support change 



Implementation 



Provide assessment workshops - start with volunteers but at some point make it a 
requirement for all teachers 

10. Require classroom application and follow-up 

1 1 . Develop models - units, assessments, etc. 

12. Have a flexible approach/examples 



Evaluation 

13. Collect evidence (to help look at student and teacher work) (See Appendix C for 
Baseline Sample Questionnaire) 

14. Encourage action 



research 



PROBING 



1. Have a vision or plan that 
supports standards-based 
assessment. 

It is critical that everyone in the 
district understands the WHY of 
the efforts to build assessment 
capacity. This can be provided 
most effectively by having a 
clearly articulated vision or plan of 
the what and why of the capacity 
building process. Linking 



In 1998, the Superintendent of Bay District Schools in Panama City, Florida laid 
the foundation for the district’s efforts to build assessment capacity. A daylong 
visioning session to promote a systemic approach for the district was held to 
build consensus that all district improvement efforts should promote Quality 
Student Achievement as the major focus. It was advised that staff 
o Plan systemically 
o Take action and experiment 
o Assess and gather data 
o Study, reflect, and evaluate using the data 
o Modify actions based on the knowledge gained, and 
o Revisit and clarify goals and purposes for all work as relevant to 
improving student achievement. 

It was evident that the superintendent was setting the example that the work at 
the district level was about a continual improvement process for student 
achievement. If an effort did not lead toward student improvement then it was 
not important and that if this attitude was prominent at all levels, district, school, 
and teacher, then a more systemic approach to improving student achievement 
would emerge. 
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assessment capacity as a driving force to improving student achievement can help focus district 
efforts. Creating and maintaining a district assessment committee (with members from various 
schools and district office staff) is an asset to keeping a district focused on a vision for change. 

2. Take on the challenge of change in secondary schools. 

Most districts have begun by having a K-12 focus for their initiative to build assessment capacity 
but have found that their efforts meet with more resistance at the secondary level. This seems to 
be particularly so in the areas of grading and 
reporting. Several districts have redesigned 
their elementary report cards but have not 
carried this revision process into secondary 
reporting. It is very important that districts 
support quality assessment and reporting 
practices at the secondary level. 

3. Define standards-based assessment, 
especially as it relates to large-scale 
(state) assessments. 

Even the most successful districts felt that 
they are involved in an ongoing struggle to 
maintain quality classroom assessment 
practices in the face of perceived pressure 
from state accountability systems to raise 
test scores (which in turn can lead teachers 
to narrow the classroom assessment 
methods used). Districts need to become 
advocates for quality assessment by publicly 
defining standards-based assessment and by promoting the appropriate use of assessment results. 
Teachers need help in understanding that standards-based assessment in the classroom does not 
mean that assessment methods should mimic the content and format of state tests. 

PLANNING 

4. Involve district leadership. 

It is clear from the districts involved in this study that a critical component is the public role of 
district leadership. The superintendent - and ideally the School Board, must be involved from the 
beginning and should make frequent public statements supporting any activities aimed at 
building teachers’ assessment capacity. It is important to keep people in key areas informed of 
any specific initiatives that may affect the capacity building process. For example, purchasing 
technology software for creating district reporting forms (cards) needs to mesh with classroom 
assessment efforts. Thus, the technology leaders and assessment leaders need to discuss and 
review the software options together. When district communication around assessment capacity 
is limited, then some efforts may be at cross-purposes. 



The Waterloo Catholic District School Board has taken on 
the high school challenge at the classroom level. The 
School Board is committed to helping students achieve the 
expectations outlined for the Ontario Catholic School 
Graduate. The District also intends to ensure that the 
assessment, evaluation, grading, and reporting practices in 
the secondary schools are consistent with policy, system 
philosophy, and mission. To do this they have developed 
guidelines that provide a common language for district 
assessment terminology and share with teachers the 
expectations that guide classroom practices to ensure 
consistency for all students. 

For example: Under “Assessment and Evaluation of 
Student Achievement of Curriculum Expectations,” 
Section 1.5 states that teachers will use ongoing 
“formative” assessment activities to monitor student 
performance and provide feedback in an effort to enhance 
and improve learning and instruction and that formative or 
diagnostic measures will not be used in determination of 
report card grades rather the data will be tracked and kept 
in folders, portfolios or recorded separately in a grade 
book. 
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5. Create time. 



The single thing that teachers need most to be able to change their practice is time to discuss, 
reflect, and plan. Thus, any attempt to build teachers’ assessment capacity must provide this 
time. There are many ways to make time available for teachers to work together but teachers 
should be asked to help determine how to do this and how their time would be monitored or 
facilitated. 



6. Include school leaders (early). 

Similar to the critical role of the 
superintendent at the district level, 
the visible involvement and 
support of interested school 
building administrators is critical 
to success at the school level. If 
administrators appear disinterested 
or question any initiative, it 
provides teachers who are looking 
for any excuse not to participate 
with exactly what they are seeking. Administrators who want teachers to implement better 
classroom assessment practices need to know what to look for when they evaluate teachers or 
provide feedback for improvement. 



Bay District Schools mandated that all principals and other school 
leaders actively participate in the required four-day assessment 
training for all teachers. Like the participating teachers, the principals 
were given an assignment to bring to the follow-up session (e.g., 
classroom observations of a particular assessment process). They also 
participated in a meeting with district staff, prior to the training, that 
focused on the goals and significance of this training. As a way of 
keeping principals focused on instructional quality, the district 
requires principals to attend monthly professional development 
sessions called PS 101 led by the Assistant Superintendent. These 
seminars explore issues in improving instruction. 



7. Ensure adequate financial resources. 

Building assessment capacity costs money - 
money for expertise and material resources, and 
mainly funding to support teacher time. A multi- 
year effort will be necessary for success so 
districts must be sure that they have the necessary 
financial resources to maintain the process. 
Districts that seek larger grants can offer mini- 
grants to schools as an incentive for making 
classroom assessment a focus for 
professional development. 



The Alcorn School District in Corinth, MS was 
able to expand their assessment emphasis beyond 
an initial pilot group of schools receiving SERVE’s 
assessment training. They wrote for and received 
grant funding from multiple sources to help cover 
additional training for teachers throughout the 
district in assessment. With this funding they were 
also able to bring in other programs that 
complemented their focus on assessment capacity 
building. 



8. Organize resources: especially 
staff, to support change. 

Organizational structures and human 
resource allocation are put in place to 
meet perceived needs at district and 
school levels. Sometimes these 
structures become fixed regardless of 
what changes are being made/attempted. 



Some districts are rethinking their use of positions. For example, Bay 
District Schools now has a Classroom Assessment Resource Teacher 
who knows what quality assessment should look like and develops 
strategies to help teachers improve. The duties of this position go 
beyond those of the more traditional testing and evaluation position 
that involves the administration and reporting of state and other 
assessments. 

In Douglas County Schools in Colorado, there is a Director of 
Assessment who works to help the district keep focused on standards 
and assessment. Aurora Public Schools in Aurora, CO also has 
placed resource teachers in each school who are there to help schools 
work with standards, assessment and instruction. 
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Several districts have changed the allocation of human resources and staffing structures to 
provide individuals whose role is dedicated to providing assistance to teachers where they are - 
both physically and pedagogically. Creating roles such as “Quality Work Facilitator” or 
“Instructional Facilitator" appears to be a very powerful way to ensure best practice and build 
assessment capacity. At the district level, it is important to have someone on staff who is an 
expert on classroom assessment and how it differs from large-scale assessment processes. 

IMPLEMENTATION 

9. Provide assessment workshops- start with volunteers but at some point make it a 
requirement for all teachers. 



One of the basic problems with building district assessment capacity is the lack of training in 
quality assessment that teachers have received. It is thus essential that teachers be provided with 
this training. The efforts of the districts involved in this study suggest that this training is most 
effective when it starts with volunteers and/or teachers identified as stars/leaders, or school based 
assessment teams. Some districts have only used this approach but if a district is truly going to 
build assessment capacity, before too long all teachers should receive assessment training. 
Districts should invest in time to create an assessment-training program for all teachers - from 
building a knowledge base for some to continued growth in this area for others. This is necessary 
so that there is a common language and common expectations in the district about how 
standards, instruction, and assessment should be implemented. 

Ken O’Connor the third author reflects 
on his experience with Pressure vs. 

Support for initiatives that lead to 
building capacity in assessment: "One 
of the most difficult decisions that a 
district has to make when it decides to 
try to ‘move ’ all of its educators to 
greater assessment literacy and more 
appropriate assessment, grading, and 
reporting practices is to determine the 
appropriate balance between pressure 
and support. To a considerable extent 
this balance depends on the culture or 
tradition of the district with regard to 
change. 

A support model is preferable in terms of developing appropriate attitudes toward the change 
but a support model alone generally results in relatively little real change. A pressure model is 
preferable if rapid change is desired but such an approach may be met by resistance and 
subversion and the change may be more apparent than real. 

Having observed a number of districts that have struggled with this dilemma it appears that the 
most effective model is one that starts with support and voluntary involvement but after the 
change effort has ‘taken root ’ moves to a pressure model with a clear requirement that 



Bay District Schools in Panama City, Florida began their 
assessment work through a grant they received from the state 
(called TARGET in the district). They set up a program within 
the district for the money to be used by the schools but only 
through an application process. The stipulation for receiving any 
money was that the school would form a team that would 
participate in the awareness training in assessment offered by the 
district. This process helped to build a cadre of trained staff for 
schools and each school could then use their money to receive 
additional resources, attend a conference, or other training 
session to continue building their own assessment literacy. 

When the district then mandated assessment training for all 
teachers (the next year) many from this initial group (after they 
were trained as trainers) were able to help in training the teachers 
and offer support and encouragement for others back at their 
school. 
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assessment, grading, and/or reporting practices will be in line with district expectations by a 
stated date preferably at least a year from the date of the announcement. Examples of this 
approach have been observed in Bay District Schools, York Region District School Board, and 
the Waterloo Catholic District School Board. ” 

10. Require classroom application and follow-up. 



Some districts have 
provided excellent basic 
assessment training but 
have been disappointed to 
find that this training has 
had little impact on what 
actually happens in the 
classroom. It is therefore 
important that training 
always include the 
expectation that teachers 
will apply what they have 
learned in the classroom. It 
is important to hold 
teachers accountable for 
using the knowledge they 
gain and for reflecting on 
what they learned. This seems to work best when teachers attend follow up sessions where they 
are required to share examples of what they did in their classroom and discuss what worked. 
When teachers get excited about how changes in their classroom practices affects student 
learning the change will be sustained. It is also important to hold administrators accountable for 
making sure they know and use the language and understand how standards, instruction, and 

assessment must link in the 
classroom. The teacher 
evaluation system in use in 
the district should be 
reviewed to ensure that 
expectations for teachers in 
the area of classroom 
assessment are clearly 
defined. 

11. Develop models - 
units, assessments, 
exemplars, etc. 

Given the lack of 
knowledge of quality 
assessment and the fact 



Bay District Schools in Florida has worked hard to incorporate the Sunshine 
State Standards and has helped their teachers become aware of using the 
standards and assessing to the standards. 

With grant money this district has developed an online curriculum resource 
center called Beacon Learning Center ( www.beaconlc.onj ~). This center has 
created a one-stop location for effective lesson plans, student activities, 
materials, resources and assessment for teachers in Florida. The district 
produced a stringent validation process for lesson plans, which includes a rubric 
for scoring the teacher work for consideration to be submitted to Beacon. 
Teachers working with this group agree that some of the best assessment 
training they have encountered has been to critique the teacher work and 
determine if it really gets at understanding and assessing the standards. 

This site is also working with SERVE staff to incorporate the assessment 
training that was used in the district into a web based format for teachers new to 
the district or as a review. This site is helping the district maintain their focus on 
building assessment capacity. 



In the Madison County School District in Huntsville, AL assessment 
literacy was built through a district assessment team who asked each 
school in their district to send a team of staff from their schools to work 
with SERVE and others over a two-year period. These groups (about 85 
staff) were trained in classroom assessment based on SERVE’s training 
and then were asked to go back to their classrooms and practice with their 
students and bring back to the continuing sessions what they learned, and 
had done with their students or schools after the training sessions. For 
some schools individual teachers concentrated on understanding and 
using rubrics correctly with their students, or just getting the whole 
process of assessment in their minds. Other schools developed assessment 
units around standards that the school had in place and ended their school 
year with a huge exhibition that shared with parents the things their 
students had done throughout the year and how it was assessed. 

What was effective about this approach was that the teams not only were 
responsible for building their own assessment literacy but also helped to 
build district assessment capacity since many of these teams taught the 
teachers at their schools what they had learned in assessment. 
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that many teachers ‘don’t know what they don’t know,’ a very powerful way to improve teacher 
knowledge and understanding is for districts to develop model units and/or assessments that are 
available for all teachers in the district. This has two powerful effects - first, teachers are able to 
see how successful quality approaches to learning and assessment are with their students, and, 
second, they have exemplars to use as guides as they develop or revise their own units. These 
models also can be used as examples for what quality work should look like for students and 
parents. 

12. Have a flexible approach 

It is clear that this is not a ‘one size fits all’ exercise - there are a number of variables involved 
and so many differences between districts that there is not one definitive recipe that will work for 
all. The lessons learned should be adapted to the conditions and culture of each district. For 
example, as regards creating time for teachers: in some districts this will work best as release 
time during the school year; in some districts pay for time spent after school may work best; 
while in many districts the most effective work will be done in the summer. 

EVALUATION 

13. Collect evidence 

In this age of accountability, in order for assessment capacity building to be sustained, school 
districts are going to have to be able to demonstrate the positive effects of quality assessment, 
especially on student achievement. It is essential that thoughtful efforts be made to collect 
evidence about how changes in classroom 
assessment impact student motivation and 
learning. 

14. Encourage action research by 
teachers 

Action research is particularly valuable 
because it provides teachers with the 
evidence they need to support their hard 
work and with the information on what 
needs to change. If there are ways for 
teachers to present to peers their findings 
about the impacts of the changes in 
classroom assessment they make (and 
receive recognition for their efforts), its 
value will be even greater. 



Roanoke Rapids and Elizabeth City/Pasquotank School 
Districts have been working for several years to create 
district assessments in fourth grade math that get at broader 
mathematical understanding. They have field-tested the 
math assessments and collected exemplars to use as 
evidence of what quality student work should look like. 

John Parker from Roanoke Rapids shares: “In our work we 
are realizing that when teachers look at student work after 
assessment training they begin to see student work as a rich 
source of information and understand the need for more 
consistency in the classroom. They begin to ask questions 
like, “What am I looking for? What is the purpose for this 
assessment? Was my criteria clearly stated to the students? 
Were there areas of bias on the assessment? How many 
samples of work are necessary for student achievement?” 



Summary of lessons learned . The lessons learned outlined above provide an organizer for a 
district to think about prior to beginning any professional development initiative. In the authors’ 
opinion, the most important aspects to think about may be how well the district: 

> Creates time 

> Requires classroom application and structure follow-up 
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> Organizes resources, especially staff, to support change 

> Collects evidence (to help look at student and teacher work). 



These four actions represent a significant departure from common practice in professional 
development (the one-shot workshop). They also directly relate to what more than 1,000 Bay 
District teachers participating in professional development in classroom assessment reported as 
important needs. Together, they suggest that engaging in learning about how to improve 
classroom assessment practices is a long-term process, not likely to happen in a single year and 
not likely to happen within the context of teachers acting in isolation. 

To educate all students to higher levels of learning around prioritized state and district standards, 
teachers will need time and support in constantly revising their instruction and assessment based 
on how well students are performing. Districts will need to help schools provide opportunities 
for teachers to continuously improve classroom assessment in a variety of ways that might 
include: 

> Collecting, reviewing and reading relevant research, 

> Searching for high quality instructional materials and assessments, 

> Talking to peers about ideas, 

> Developing and trying out new assessment methods that give them better information 
about how to help students improve, 

> Reflecting on the quality of work they assign and making it more rigorous and 
challenging, 

> Exploring the role of rubrics in helping students internalize expectations for quality work, 

> Searching or observing others use practices like peer or student self-assessment in ways 
that develop student responsibility for learning, 

> Experimenting with how to document student learning on classroom goals, 

> Examining grading practices in light of state and district standards. 

It is increasingly clear as one reads through the kinds of ways teachers might go about improving 
classroom assessment practice that professional development in classroom assessment is part of a 
bigger picture of how adult learning is structured and the extent to which continuous 
improvement of practice is expected of teachers. 



Building a Framework for District Strategies to Leverage Teacher Change 



In the previous section we identified a cycle of planning fc 
continuous improvement that emerged from experiences v 
districts engaged in providing teachers’ professional 
development in classroom assessment. In this section, we 
will explore how districts might reflect on their professional 
development efforts within the context of a broader 
framework. 




i 



Building Teacher Capacity 

The Thinking Cycle 
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There is no road to improved student motivation and achievement that does not pass through the 
classroom. When district leaders are asked what it will take to improve teaching, they mention 
more or better professional development (Spillane, 2000; Duggan & Holmes, 2000). Yet at the 
same time, research on professional development suggests that traditional workshop approaches 
have had little impact on teacher practice on a large scale. “Previous studies of school districts 
professional development programs offer a less than optimistic account: professional 
development is firmly rooted in the training paradigm and focused on the individual teacher, 
typically with short-term activities that involve little follow-up. They are market-oriented and 
menu-driven and have little coherence or coordination” (Spillane, 2000, p. 1). 

Duggan and Holmes (2000) conclude, “Professional development needs to be improved to 
further develop the educators we need in our classrooms, schools, and in leadership positions. 
This means giving teachers and principals the opportunity to work together, network, look at 
student work, and learn from each other. It means giving them the training they need to fully 
implement the many policies of standards-driven reform. There are standards for high-quality 
professional development, and yet much money is squandered every year on one-shot workshops 
that have little effect on instruction” (Duggan & Holmes, 2000, p.2). 

Here, we introduce a framework that may help districts conceptualize and critique their direct 
and indirect 
strategies for the 
continuous 
improvement of 
classroom 
practices. The 
graphic outlines 
the four major 
categories for 
which districts 
might consider 
outlining 
strategies. Once 
the strategies, 
used in each of 
the four 

categories shown, 
are described then 
a district should 
be able to analyze 
the likelihood of 
achieving success. 

Notice that 
Building Teacher 
Capacity is the 

first category and is in the center of this triangle as we see it as the centerpiece of district 
improvement efforts. Therefore, we will discuss this category in more depth using a district 
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example and offer a few basic strategies in the other three categories as an example of how the 
graphic may help a district think about their particular effort. 

1) Efforts to Build Teacher Capacity: 

Increasingly, in the last several years, some districts are addressing the need to build educator 
capacity to improve their classroom practice and are implementing more powerful professional 
development strategies for improving instruction. For example, Elmore and Burney (1998) 
describe Community District #2 in New York City as engaged in a long-term continuous 
improvement process rather than the more typical special projects approach to professional 
development. The project approach to improvement is time-bound, for example for a year, 
disconnected from previous staff development efforts, and used to introduce teachers to a new 
concept, initiative, or set of materials, with little evaluation of teacher follow-through in the 
classroom. In contrast, they describe Community District #2 as having a “well-worked out 
system-wide strategy for influencing classroom instruction”. The key elements to the strategy 
are: 

1) A focus on specific content areas over time (like literacy) 

2) Strong investments in various forms of professional development structured to provide 
support for engaging in new forms of instructional practice 

3) Development of strong professional networks across schools to reduce isolation and 
develop a district-wide culture 

4) Implementation of individual accountability processes such that principals and teachers 
are held responsible for quality learning environments 

5) Structures for negotiation between schools and the district leaders on expectations for 
performance given individual differences between schools 

Other points noted about District #2’s efforts at building capacity: 

■ The district spends 4% of its total budget on professional development for teachers and 
principals. Professional development is a general strategy for improvement rather than a 
department or isolated administrative function. The following statements are the five 
strategies used: 

a) A Professional Development Laboratory 

b) Instructional Consulting Services delivered to individual teachers and school 
teams 

c) Inter- visitations and Peer Networks 

d) Off-site Training (e.g., designed to introduce successively larger numbers of 
teachers to a central concept of teaching in a content area) 

e) Oversight and Principal Site Visits (e.g., district staff spend two days per week in 
schools monitoring progress on instructional improvement) 

■ Principals are viewed as key players. The district trains and retains principals based on 
their ability to function as instructional leader, which includes recruiting, nurturing, and 
counseling-out of teachers when needed. Accountability for school improvement 
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involves an ongoing negotiation between principals who understand their school’s unique 
characteristics and district leaders who want to see results from all schools. 

The advantages of closely reading a case report like that of Community District #2 is that district 
leaders can compare and contrast their professional development strategies to those used by 
Community District #2. Do they have a set of strategies that seem more or less powerful in 
terms of impacting instructional improvement (building teacher capacity) over time? 

Case studies of districts involved in standards-based reform have pointed out other areas that are 
indirect influences on teachers’ classroom practice (Laboratory Network Program, 2000; Goertz 
and Massell, 1 999). It is important that districts analyze how these other areas support or 
perhaps hinder efforts at building educator capacity to improve the quality of the classroom. 
Below we share the other three categories and a list of some strategies that can indirectly shape 
classroom teaching and learning. 

2) Setting the Conditions for Continuous Improvement 

These strategies represent the kinds of things district leaders engage in that set the parameters for 
organizational capacity building. 

■ Defining high academic standards; identifying broad learner outcomes 

■ Engaging the public in strategic planning 

■ Providing curriculum guides and instructional materials 

■ Finding external partners and resources 

■ Communicating a consistent vision 

■ Using district-wide committees to develop teacher leaders 

3) Supporting School Improvement Processes 

These strategies include the things districts do to empower schools as learning organizations in 
their own right. 

■ Encourage schools to define Standards of Classroom Practice 

■ Create more time and resources for teacher collaboration around “working on the work” 

■ Have regular informal school reviews 

B Provide district coaches/assistance for struggling schools 

■ Establish professional development committees at schools who are accountable for how 
adult learning is structured 

4) Monitoring the Results (process and outcomes) 

These strategies include ways districts can monitor district, school, and student progress on 
important dimensions. 

■ Supplement state tests with district assessments 

■ Require teachers to assess certain outcomes as part of a portfolio process 
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■ Find ways to assess the quality of instruction 

■ Develop electronic portfolios to help teachers manage data 

We do not show the above strategies as the “right set” but rather as an example of how a district 
could outline its direct and indirect strategies for impacting classroom quality, and therefore 
begin to examine the package in terms of pieces missing and likelihood of their resulting in 
instructional improvement. 

IMPLICATIONS FOR FUTURE RESEARCH 



“ There has been a strong tendency in recent federal and state policy initiatives to by-pass or 
ignore districts ’ role in the change process.... In many ways, districts are the major source of 
capacity-building for schools, structuring, providing or controlling access to professional 
development, curriculum, and new instructional ideas, more and qualified staff, relationships 
with external agents, and so on.” (pg. 17, Goertz and Massed, 1999) 

Working in an organization that straddles the line between the worlds of practice and research, it 
is important to reflect on what research is needed to help districts make informed decisions about 
professional development. There is research that makes a strong case for the relationship 
between effective classroom assessment practices and positive student outcomes (Crooks, 1988; 
Black and Wiliam, 1998). But, we have very little research that helps districts lay out a strategy 
for improving classroom assessment practices over a sustained period of time. The problem in 
helping teachers improve in their use of assessment in the classroom is no different from the 
bigger picture of teacher change as described by Richard Elmore. 

“A significant body of circumstantial evidence points to a deep, systemic incapacity of American 
schools, and the practitioners who work in them, to incorporate, develop, and extend new ideas 
about teaching and learning in anything but a fraction of schools and classrooms. This 
incapacity, I will argue, is rooted primarily in the incentive structures in which teachers and 
administrators work. Therefore, solving the problem of scale means substantially changing 
these incentive structures ” (Elmore, 1996, p. 294). 

The two quotes above, along with our field-based experiences, lead us to conclude that districts 
are key to any effort to improve teacher practice in assessment on a large scale but the obstacles 
loom large. In reflecting on our past experiences with classroom assessment and on the research 
emerging on the district role in professional development, we suggest several areas for future 
study. 

1) How can professional development most effectively be provided to improve classroom 
assessment practices? Several reviews of the literature on the use of assessment in classrooms 
conclude there are substantial problems with current practice (Crooks, 1988; Black and Wiliam, 
1998; Shepard, 2000). Black and Wiliam summarize some of the problems with classroom 
assessment as: 
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1) Classroom assessment practices generally focus more on superficial or rote learning, 
concentrating on recall of isolated details, which students soon forget. 

2) Teachers do not review their assessments/assignments or get peers to review them so 
there is little critical reflection on what is being assessed and why. 

3) The grading aspect of assessment is overemphasized and the learning or improvement 
purpose of assessment is underemphasized. 

Shepard (2000), based on a review of the role of classroom assessment in teaching and learning, 
suggests that the following changes should be encouraged: 

1) Changing the nature of the assessment conversations teachers have with students such 
that students develop greater knowledge and responsibility for learning goals. 

2) Assessing students’ prior knowledge and using that information in planning better 
instruction to meet their needs and match their interests. 

3) Giving students feedback in ways that go beyond grades such that they are helped to 
understand what quality work or thinking looks like. 

4) Getting clearer about the explicit criteria for open-ended/performance tasks and involving 
students in self-assessing. 

5) Using information from students to evaluate and improve teaching strategies. 

Stiggins (1999) has been perhaps the most ardent advocate of the need for “assessment literacy”. 
He advocates a “learning team” approach to engaging teachers in learning about classroom 
assessment. Others argue that professional development should primarily be structured within 
the context of the disciplines/content taught and not as separate methodologies (e.g., cooperative 
learning, classroom management, technology, assessment) that the teacher then has to figure out 
how to fit into the content area taught. 

The results of two curriculum intervention studies (Wilson & Sloane, 2000; White & 
Fredericksen, 1998) seem to indicate that teachers’ work on. curriculum changes (implementing 
new instructional and assessment materials in a course or a unit) had significant impact on 
student achievement only in conditions where there were opportunities for discussing with peers 
the quality of student work that results. That is, just giving teachers good performance tasks 
(assessments) to use in their course may not necessarily lead to better learning unless there is 
involvement with other teachers or external experts on how to use the assessments, decipher 
what the assessments say about student learning and change instruction to result in better student 
performance. 

More research is needed on how best to structure professional development on classroom 
assessment. For example, should districts 

o Offer year-long, job-embedded courses on classroom assessment, 
o Organize and support teacher “learning teams” who engage in self-study, 
o Provide quality assessment materials to teachers along with opportunities to talk with 
other teachers about student results on those assessments? 
o Mandate teacher participation in the initiatives involving assessment? 
o Offer individualized feedback to teachers on how they can improve their use of 
assessment? 
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2) Is engaging teachers in working on the quality of work given students a better wav to focus 
district energies and discussions than the broader goal of improved classroom assessment? 

One of the problems with tackling the improvement of classroom assessment as a district goal is 
that it is so overwhelming. One review of the research on teacher change suggested that 
sustained change in teaching practice is unlikely if either the suggested change is too trivial or 
requires radical or fundamental changes in a short period of time (Gersten et al, 1997). The 
authors suggest, for example, that the changes described by the National Council of Teachers of 
Mathematics (NCTM) standards may be too removed from current practice and too unspecified 
to expect of teachers with little support. The same may perhaps be said for the kinds of changes 
outlined by Shepard above that are needed in classroom assessment. Thus, care must be given to 
how suggested improvements to practice are described and staged for teachers. 

The reflections on Community District #2’s experiences are interesting in that the authors report 
that, initially, professional development was focused on key instructional strategies or perhaps, 
identified “best practices” (Learning Research and Development Center, 1999). Over time, 
district administrators, professional developers, and principals increasingly began to focus on 
“high quality student work”. The depth and quality of student work has become the language 
used in talking about improving classrooms. 

“ System-level administrators constantly referred to the fact that the first thing they look for 
when they visit schools and classrooms is evidence that students are working at high levels of 
effort on important aspects of academic content, that student work is prominently displayed, 
discussed and analyzed in schools, and that students and teachers are able to make judgments in 
their own daily work about whether they are engaged in important and challenging work. 
District administrators, for example, consistently ask students in the classrooms they are visiting 
whether they are working hard and whether they are interested in or bored by what they are 
doing" (LRDC, 1999, p. 21). 

Validating this emerging focus on the quality of work, in a separate study, Newman, Bryk, and 
Nagaoka, (2001) found evidence that students who received assignments that involved more 
rigorous intellectual work achieved higher performance on standardized tests than students who 
received lower quality assignments. More research is needed on how concentrated efforts by 
districts to engage teachers in improving the quality of work unfold and whether focusing efforts 
in this concrete way lead to improvements in the kind of best practices in classroom assessment 
envisioned by Stiggins and Shepard. 
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Appendix A 



1 . Bay District Story on Building Assessment Capacity 

2. Baseline Data from Volunteer Schools 

3. Summary Sheet of TARGET Assessment Training Data 

4. Teacher Satisfaction with Target Training 

5. Teacher Comments about Impacts of Training 

6. Teachers’ Responses about Areas Needing Assistance 
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Districts can work collaboratively with universities, labs, or other districts to 
build assessment capacity - below we share a partnership example from one 

of our research and development sites: 

Reflections of a District’s Growth - From the Field 

By Patricia Schenck, 

Assessment Resource Teacher in Bay District (850-872-4352) 

Past Accomplishments of our Collaborative Research and Development 

Since the advent of the Florida Sunshine State Standards in 1995, Bay District has been working 
toward systemic standards-based reform that brings curriculum, instructional strategies, and 
classroom assessment practices into alignment with state standards. During this time, SERVE 
has served a vital role as our educational partner. This partnership has not only allowed us to 
implement intensive research and development projects within three district schools, but also 
enabled us to implement in-service training for all district teachers - over 2000 teachers and 
administrators to date. These experiences have prepared district teachers to create quality 
classroom assessments and implement an instructional and assessment system in classrooms that 
supports state standards. Throughout this process, SERVE has helped up with technical support, 
developed training models, facilitated planning and goal setting meetings, aided in our grant 
writing efforts, provided data-analysis and evaluation services, and supported classroom 
implementation work. With this partnership our district has been able to view our progress in a 
more systemic process. 

It is clear that true systemic reform is a long-term process. As a district, our goal has been to 
show continuous incremental improvement that over time will lead to meaningful and 
sustainable change. In the standards-based reform arena, our efforts and experiences over the 
past five years have placed our district in a unique position. Our teachers are now prepared to 
create tangible, performance-based models of standards implementation. This work could not 
have been accomplished five years ago. Five years ago, our district needed to begin building 
capacity for improvement somewhere, and with SERVE’s help, we began with the new Sunshine 
State Standards and Classroom Assessment. We are proud of the intensive work we have done 
here, but at the same time we realize that our work has just begun if we want to continue to 
improve student achievement. 

What we have accomplished over the past five years: 

Our vision for our teachers: 

“For each standard, we want teachers to have a clear picture of the achievement 
target and have a shared understanding of the student performance needed to meet 
that standard.” 
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Part one of our Research and Development Project - the early years; 
Standards Work - Phase I 

■ Professional Development 

-Connections Training (all staff) 

-Alignment of CAI to New State Standards 

■ Leadership 

-PS 101 (Principals training sessions) 

-AP Research Group 
-Leadership Institutes 
-SERVE Leads 

■ Research & Development 
-Beacon (Learning Web site) 

-Site Analysis of Standards Implementation 

Phase I - Lessons Learned 

■ Districts should not work on parts in isolation 

■ All parts must be considered as a whole 

■ Teachers must “Unpack the Standards” 

■ Standards must match assessments and instructional strategies 
Some of our accomplishments with building assessment capacity: 

■ Developed a district vision tied to student achievement 

■ Supported professional development tied to the vision 

■ Developed district policy and guidelines around assessment or other initiatives 

■ Conducted research and development during the process and used the data to make 
informed decisions 

■ Obtained resources and supported collaboration at all levels within the district 

■ Created a standards implementation model 



Reseateh Proi&t OwWta I 




Part II of our Research and Development 
work - past two years _ 

Assessment work - Phase II 
B Professional Development 

-Target/SERVE Quality Assessment 
-Creating Writers 
-Reading Framework 
-Grading & Reporting 
■ Leadership 

-Assessment Staff / Liaison 
-Assessment School Teams 
-District Assessment Committee 
-ASCD Assessment Consortium 
-SERVE 

B Research & Development 

-Alternative Assessment Implementation 
-Grading & Reporting 
-Standards-Based Reporting 



Restate!! NjetfOwtw-ta II 
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Assessment Lessons Learned 

■ Quality Assessments determine student achievement 

■ Teachers need assistance in understanding assessment methods (D, F, & S) 

■ Changes in assessments will lead to changes in grading practices and reporting methods 

■ District assessment guidelines must be developed for consistency 



With this collaborative relationship, SERVE has had a significant impact not only on our 
district’s teachers but also on our students. State assessment data shows steady increases in 
student achievement. Our state accountability system recognized six A schools in our district this 
year. We did not have any A schools last year. 

Next Steps for Bay District 

We recognize that the ultimate indicator of student performance is the production of educated 
citizens that are prepared to enter the workforce or be successful in higher education. Our 
graduation rate, or percent of students who complete high school in a four-year period, is cause 
for concern. Our most recent data shows a graduation rate of only 56%. This places our district 
58 th of 67 districts in the state of Florida. In addition, we have a significant population of at-risk 
students. Although Bay District cannot be considered a low-performing district, each of our 
schools serves a population of low-performing students. Addressing the needs of these students is 
currently our number one district priority and a clear area of need. 

Currently, we have identified our area of greatest need and will need SERVE’s help to continue 
our work. We have used SERVE’s graphic below as our district guide when thinking about 
creating a standards-based system; the darkest gray area (Monitoring and Feedback) is where we 
need to continue our work. We have worked diligently to build capacity in 1) Standards and 2) 
Assessment over the past five years, but now we must begin to build on the 3) Teaching and 
Learning Strategies to monitor what is happening with standards and assessment in the 
classroom. 




So, clearly our next steps include: 

■ Develop a Standards-Implementation Plan for the District 

■ Design a research study 

■ Create a Standards Implementation Plan Team to develop proficiency models 

■ Implement proficiency models in a feeder school pattern 

■ Monitor process for effectiveness 
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■ Collect feedback from participants 

■ Assess the overall impacts 
-Proficiency rating of students 
-Implementation of SIP 

-Policy changes (based on Classroom Assessment 
Guidelines) 

-Grading & reporting changes 
Benefits of our Partnership with our Regional Educational Laboratory 

We know that SERVE has benefited from our collaborative relationship over the past five years. 
They have been able to develop training materials, look at processes, and build a trainer network 
in Bay District and use theses resources in other districts or areas of their work. We are proud to 
be able to send our teachers or staff to conferences to present what we have done here or into 
others districts to conduct training or help in planning for systemic change. We have been able to 
share our expertise and resources, developed as a result of our work with SERVE, with many 
visitors SERVE sends to our district. We have also been asked to host the 2001 Spring ASCD 
Assessment Consortium conference in Bay District to share with international school district 
teams and others our work in the area of standards and assessment and have those individuals 
provide us feedback on next steps. 

Lessons we have learned about building capacity 

■ Maintain a focus on the vision and maintain a clear vision 

■ Systemic change takes time 

■ Focus on one step at a time 

■ Interim decisions should be based on formative data and broad-based data is essential to 
make good decisions 

■ Teachers must have a well-developed support system 

■ Invest in people - professional development is 
important and develop customized training 

■ Develop leadership for the implementation 

■ Collecting research is vital - it supports follow-up 
and application at the classroom level 

■ Beg, borrow, and steal - adapt established models to fit your needs 

■ Locate and provide resources to make the implementation a success 

■ Recognize that reform takes long-term dedication 

■ Working collaboratively with others is vital 

■ Looking at work (student and teacher) is necessary to know what is going on in the 
classroom 

" Realize that you are NEVER finished -there are no ‘quick fixes’ we must trust our 
feedback loop for learning 






Note the italicized bullets are 
the same recommendations 
that O’Connor and SERVE 
noted as significant because 
of the classroom impact. 



Fora copy of Bay 
District’s Guidelines for 
Classroom Assessment 
contact: 

Bay District Schools 
850-872-4352 
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Bata From Volunteer 

(May 1996) 



immu 



Column A 

Percent of teachers who report “less than fully understand ” 

Column B 

Percent of teacher who report, “using successfully” 



Category Patronis Rosenwald Rutherford 



METHODS 


A 


B 


A 


B 


A 


B 


Notebooks 


74% 


11% 


10% 


33% 


23% 


47% 


Performance Tasks 


70% 


19% 


33% 


33% 


28% 


43% 


Observations 


45% 


19% 


18% 


38% 


34% 


34% 


Videos 


66% 


11% 


27% 


24% 


34% 


39% 


Cooperative Groupwork 


39% 


32% 


18% 


, 43% 


37% 


37% 


Exhibitions/Projects 


47% 


40% 


27% 


40% 


36% 


35% 


Open-ended Questions 


55% 


17% 


59% 


29% 


42% 1 


32% 


Debates/Skits/Oral Performances 


63% 


14% 


27% 


20% 


44% 


21% 


Graphic Organizers 


68% 


14% 


23% 


43% 


45% 


25% 


Logs/Journals 


50% 


19% 


23% 


24% 


46% 


24% 


Portfolios 


55% 


22% 


5% 


29% 


50% 


18% 


Interviews 


74% 


14% 


41% 


20% 


53% 


13% 


Anecdotal Records 


71% 


11% 


50% 


14% 


67% 


11% 


ASSESSING QUALITY 


A 


B 


A 


B 


A 


B 


Rubrics 


84% 


11% 


24% 


38% 


77% 


17% 


Critiques of student work samples 


89% 


3% 


39% 


14% 


67% 


18% 


Peer feedback 


78% 


3% 


29% 


29% 


60% ! 


16% 


Student self-assessment 


78% 


9% 


24% 


19% 


58% 


17% 


Student-led parent conferences 


86% 


0% 


62% 


14% 


82% 


4% 


Standards/benchmarks 


78% 


9% 


57% 


14% 


63% : 


27% 



Baseline data in Bay District Schools - school field test sites for Phase 1 of SERVE Project - Building Assessment Capacity 
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TARGET ASSESSMENT TRAINING INITIATIVE - DATA SUMMARY 

Bay District Schools 1999-2000 



Topic Targeted 
For Teacher Perceptions 


Number 

Responses 


Findings 


Teacher satisfaction for 4-day TARGET 
Training Initiative for Bay District on 
Classroom Assessment 


1,344 


74% (Excellent/above average rating) 
26% (Average or below rating) 


Assessment tasks created and used with 
students - collected and reviewed by 
district staff 


947 


59% (submitted performance tasks for review) 
36% (submitted various other forms of alternative 
assessments for review) 

5% (did not what assessment type they submitted) 


Specific content areas addressed by 
assessment process for submitted task. 


620 


39% Language Arts 
15% Science 

15% Vocational Education 
31% Total other content areas 


Percent of teachers reporting this 
assessment process different from their 
traditional practice. 


1,044 


74% (reflected change in practice) 

36% (were already using some alternative assessments) 


Specific area noted as different in what 
the teacher did or used? 


690 


40% (reported use of rubrics as major difference) 

28 % (reported seeing assessment as a process and not just 
as a test or thing done to students) 

32% (other differences included student involvement, 
standards more important, teacher collaboration, and 
assessment as more important) 


Student reaction to teacher change or 
assessment process used. 


1,022 


87% (reported students responded very positively) 

13% (reported various comments - students more 
engaged, increased participation, students also thought 
there was more worked involved) 


Teacher assistance needed to sustain 
assessment knowledge and continued 
learning. 


1,024 


80% (reported needing additional assistance with 
resources, support, and time as important for 64% of this 
group) 

20% (reported no need for additional help - district staff 
speculated this group contains few who are already good 
in the use of assessment and others who do not know 
enough to know what they need). 


Overall Analysis 

After reviewing teacher perception 
data and submitted tasks. ... 
considerations for next phase of 
assessment work within district 


1 . Training motivated teachers to try something different or continue 
learning about assessment 

2. Use of rubrics was limited - teachers identified this as a need 

3. Very few teacher reported using rubrics to provide feedback - little seem 
to understand the use of diagnostic, formative and summative 
assessment measures or needs 

4. Need to find ways to encourage schools and staff to find time for 
collaborative working on the work 

5. Assessment still may not be view as a process - more just a thing - need 
to find a district explanation for what ‘quality’ assessment should look 
like. 

6. Assessments used did seem to change - however, the way the 
assessments were graded and the use of those grades are in question. 
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Teacher Satisfaction Rating for TARGET Training 

1997-1999 
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assaacs 



Assistance Needed by Teachers 

(n=1,024) 

Bay District Schools, Florida 





■ More Planning/Practice Time 

■ Help with Rubrics 

■ Resources & Support 

□ No Additional Help Needed 
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1 . Interview Protocol 

2. Table of Districts Interviewed 
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Interview Protocol 

Building District Classroom Assessment Capacity Survey 

Name of person/s completing this survey: 

Position in district: 

District name: 

Size of district by student number: 

Size of district in square miles (kilometers): 

Number of high schools middle schools elementary schools 

Percent of minority populations served: • 

Location is: urban suburban rural 



Please answer the following questions by writing complete responses based on 
experiences within your district. Please do not limit your response to just the space 
provided. If you need additional space please finish your answer on the back of the 
page. 

1. Most states have developed standards to describe for districts, schools, and teachers the 
kinds of skills and knowledge students should be developing. In many cases, these 
standards, and the associated state tests, have raised expectations for achievement by all 
students. Districts, schools, and teachers, for the most part, are left on their own to figure 
out how to teach and assess in ways that will increase student achievement aligned to the 
standards. What has your district done (at the district level) to attempt to move students 
towards higher levels of achievement? 

2. Recognizing that improvement is a long-term process, how would you characterize 
your success as a district in moving students toward higher levels of achievement? 

Which district strategies have been successful and why have they been successful? 

3. What are the biggest barriers you have encountered so far in moving as a district 
towards higher achievement for all students? 

4. Knowing that the long-term process for creating change should include attention at the 
district level to curriculum, assessment, and instruction, the focus of the questions now 
will mainly be on classroom assessment. 

4. 1 As you began to work toward improving student achievement related to 
standards, what changes did you feel were most needed in student assessment at 
the school, and district level? 

4.2 As you began to work toward improving student achievement related to 
standards, what changes did you feel were most needed in student assessment at 
the classroom level? In other words, what do teachers have to do differently? 

5. What has your district done to build teacher’s and principal’s understanding of the role 
of assessment as an important part of the learning process in the classroom? 

6. How important is building teacher’s and principals’ skills and knowledge in 
assessment in your district plan to improve student achievement? 
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7. Building assessment capacity has been and is an important part of your district’s effort 
(plan) to improve student achievement. 

7.2 Who was or is in charge of the professional development component of 
building teacher and principal’s assessment capacity? 

7.3 How did this leadership impact on the process of building assessment 
capacity? 

7.4 What components were important in the process of building assessment 
capacity? For example: 

■ District Size: 

■ Planning: (Did you have a plan? How was it developed? Is it part of a district 
strategic planning initiative? If yes, where does it fit into the strategic plan?) 

■ Target Group/s: (volunteers, applicants, all, demo schools, school assessment 
committee/teams, subject teams, etc.?) 

■ Strategies: (Same strategies for all or different for elementary, middle, and 
secondary? Were the strategies linked to feeder school groups/families of schools? 
What buy in mechanisms were used? Was there any link to teacher evaluation?) 

7.5 How has the building assessment capacity process been supported financially? 

■ For example, district staff development funds, grant money, or other sources 

7.6 Has an outside agency played a role and how has it helped? 

7.7 Were adequate resources provided? In particular, were teachers provided with 
enough time to get beyond initial training to real application? If yes, how? 

7.8 How are you ensuring that the building assessment capacity effort is 
maintained or expanded? 

8. What evidence do you have that your efforts have built assessment capacity for each of 
the following groups? 

• Teachers 

• Students 

• Parents 

• Building administrators 

• Central office personnel 

9. What have you learned from your efforts to build assessment capacity? 

9. 1 What worked? 

9.2 What did not work? 

9.3 What considerations should a district address prior to implementing efforts to 
build assessment capacity? 

10. What role did district policies (approved by the School Board) play in the process of 
building assessment capacity to improve student achievement? 

11.1 Is ‘teaching to the state test’ seen as a narrowing process that limits what teachers 
feel they can do in the classroom in your district? 

1 1 .2 What have you done in your district to keep the focus on the quality of student 
learning in general and not just on better test scores? 

12. What ideas do you have about how SERVE could be of use in helping districts 
develop the role of assessment in helping all students achieve at higher levels? 
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Code # (birth date) School: 

Grade Level Subject: 



There is a collection of terms associated with assessment practices. As a way of better understanding where teachers are 
in this school, we would like you to answer a few questions about these key terms 



UNDERSTANDING 


Never heard of 
it 


1 Heard of it but 
1 know little about it 


I Partially 

understand it 


1 Fully understand it 


1 Could teach others about 1 
! it 




*.> ?. -Zip * 






1. Open-ended questions 












2. Performance tests 

(Labs, manipulates/ students 
demonstrate a skill) 












3. Graphic Organizers (concept maps, 
flow charts, etc. 












4. Logs/Joumals 












5. Exhibitions/projects 












6. Debates/Skits/Other Oral 
performances 












7. Notebooks (An organized collection 
of all work) 












8. Portfolios (A collection of selected 
student work) 












9. Cooperative group work 












10. Anecdotal records (Narratives) 












11. Interviews 












12. Observations 












13. Videos 




_______ 








1. Rubrics 












2. Critiques of student work samples (Anchor papers) 












3. Peer feedback 












4. Student self-assessment 












5. Student-led parent conferences 












6. Standards/ Bench marks 













USE 


Never used it, have 
no interest in using 
it 


would like to 
use, but need 
more 

information 


Tried briefly, but 
discontinued or 
use rarely 


Currently using, 
but need some 
feedback/help 


Using with good success 












1. Open-ended questions 












2. Performance tests 

(Labs, manipulatives/ students 
demonstrate a skill) 












3. Graphic Organizers (concept maps, 
flow charts, etc. 












4. Logs/Joumals 












5. Exhibitions/ projects 












6. Debates/Skits/Other Oral 
performances 












7. Notebooks (An organized collection 
of all work) 












8. Portfolios (A collection of selected 
student work) 












9. Cooperative group work 












10. Anecdotal records (Narratives) 












11. Interviews 












12. Observations 












13. Videos 
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1. Rubrics 












2. Critiques of student work samples (Anchor papers) 












3. Peer feedback 












4. Student self-assessment 












5. Student-led parent conferences 












6. Standards/Benchmarks 
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Dimensions of Classroom Assessment 



u 

.a 

a 

a 

a 

◄ 



O 

ERIC 



CO ^ 
CD 

m ■ 4 - J 

S 

.y cn 
*0 

•- <u 

^ JZ 
fM -*-» 
? C 

E 

O Q) 



j" 1 a) 

< £ 



C71 ±|| 

II 

ru 

a) 



cu 

c 

cn 



3 ro 

c Si 
— o 
^ u 

js e 

D g 
o E 

^ <u 

4~» lu. 

fD fD 

£ 3 

c £ 

.2 

Id 

£ -c= 
o -M 
CX CO 

5 I J3 

4 -» <L) c 

b. e I 

to LO £ 

O) * rD 
£ ru 
O u w 

u o o 
a 5 V 

■g* CU 

i<£ 

c (t ^ 

01 
aj =6 

_<y £ "o 

y c 'e 

u 0 a> 

>“ E £ 

0 £ .c 

£1 jy £ 

ro 

3 -C 3 

C ±i O 

1 5 > 
O c c 
U <D fO 
ai p ai 

£ I E 

<- CD ^ 

O^P 



J 3 

5 § 

"O .2 
O ^ 

E 



a; 

1 

CX 

E 



c 
a; 

E 

CO 
CO 

a; 

10 
10 o 
ro 4-» 

a; "a 
to c 

3 (D 
> a; 

*1 

it 



in 



ro 



CM 



to 

CD 

T3 

2 

cn 

43 

c 

CD 

T3 

3 

to 

CD 

> 

*cn 



4-1 c 

E g 

CD E 

E £ 

ft .2 

CD -C 
to u 

55 

CD g 
CO g 
3 Q- 
CD 

_>* t- 

trj O 

i? 

HH CD 



CD 

> 

CO 

ro 

4-J 

c 

CD 

E CO 

to CD 
to CO 
a> to 
to CD 
to u 
ro o 

“O CL 

ro v 

C 2 
.2 ro 
-H cx 

3 W 

2tr 

— CD 
CD 



~ £ 
to t 
»-h -a 



in 



ro 



CM 



■a 

CD 

JO 

2 

i_ 

CD 
• 4 -» 
C 

to 

ro 

c 

o 

tJ 

3 

&. 

t* 

C 



“O P 



ro 



ro 

o. 

CD 

to 



CD 

E * 

to 4 -J 
to ^=L 



a « 
!8i 

CD 



CD TD 
to C 
CD 



to 

c 

to O 

l§ 

o>£ 

E - 

.2 j* 

|i 

3 2 
u c 

J? 

*l_ 

4-» — 

to CD 
X) a5 

l_ > 

o o 

CD CD 
■£ -C 

ro 4-i 
to CD 
c 2 

CD > 
> 2 
C9 cx 



J3 

c 

CD 

■a 

to ro 

2 £ 
<y ro 

c ^ 
2 ro 

a g 

>■ 
>- CD 
-O _c 

"O tJ 

CD 

2 -C 

|l 

> c 
tn 

o <u 

E ro 



in 



n 



CM 



ro 

2 .* 

^ 0 

2 

°-tn « 

8 § * 
XJ fc 

5 .52 
°* c 
E .2 

2 tJ 

3 2 

3 .2 
U ^ CD 



CD 

in ro 

>- 
X) 

4-J 

3 
O 

!2 
’CD 



o 

§ 

CD 

E 

£ 



o 

CD 

Id 2 
CD 
> 
o 



to 

c 

CD 

> 

b 



a; •= 



3 

« 

3 

i_ 

to 

CD 



> 

-Q 

■a 

CD 

"O 

3 

cn 



c 

o 

4-» 

c 

CD 
CX 

to 0 
.55 ‘to 

ro 5 



« 



fD 

C 

o 

t 5 

3 

L. 

to 

c 



cn 

c 

To 

u 

CD 



CD 

CD 

"O 

■a 

o 

o 

cn 



cn 

S.& 

cn c 
c o 



fe g 

CD C 



< — CD 



in 



n 



CM 



CD 

Nf 

cl 

o E 

si 

CD w 
CX ^ 

“ to 
.55 ro 

CD ^ 

Eg. 

c <U T3 
O ^ 3 

'§ |E 

Ego 

-H O U 

}2 U - 
.2 ■ — * ro 

o.2 1 

CD 
CD 

"°£ , 
“O — CD 

8S| 

ro 

< U D 



CD 

> 

CD 



CX 

3 

o <U 

XI W 

2 </> 
CD "O 

s5 

3 4~> 
no <D 

*> E 

Y S 

•- CD 
M- c 

0 I 



> 

CD 

> 

‘to 

I 



> to CD 
CD 

CD to "a 
•£r to CD 
cd ^ 55 

> CD J 1 

« to 2? 

o c fD 
^ 0)^ 

c **• ro 

ro ^ Q- 

^ "C J-> 

3 0 2 
I/) ?tc 



in 



m 



CM 



43 

43" 

CD 

CD 

-C 

J2 



CD 

4-> 

CX 

cn 2 

.E -5 

r- 

td g 
c o 
ro -q 

to 55 
E 

ro -o 

o 2 

J CL O 

■S.O I 

|I5 

£ E ro 

E -c 2 

ro OK3 

roc. 

w o VI 

-g £ 1 * 



CO 



. CD 
O 3 
C O’ 



to 

CD 

w 

3 

O’ 

B 

to 

<u 



I 

CL 

3 



O 

to 

Q) 

CX 

E 

CD 

X 

CD 



"O 
CD 

t! 

CD 
CX 

X T3 

CD c 

55 cd 
4 -» fD 
CD ’C 
-C CD 

5S 

U 

8^ 
_Q 
CD 



to 

C 

o 



CD 

-2 3 -X 

fli 

3 § O 

43 O 2 J 

CO tfc 01 



in 



ro 



fM 



cn 

3 

2 

£ T3 

XI £ 

£ ro 

U CL 

ro c 

9-g 

CD u 

55 2 

4-1 -X 
CD 

! 5 

4_j l- 

o 

-Q CD 

* -o 

C CD 

8 8 
a fi 

c to 
CD CD 
■o "O 

3 [D 

4-J i- 

co cn 



2 

CD 



■o _X 
C ‘ 



"2 w 

8 8 

?s 

£ ro 

CD CD 
*C Q- 

£ | 

u 8 

J ^ 

S <= 



o 

5 

c 

5 

o 

1_ 

‘CD 



CD 



O 

CD 

U 

C 

CD 

E 



43 

I O 

3% 

CO CX 



cn 

£ to 

C CD 

JS c 

<D o 
<D 2 



in 



ro 



CM 



to 

c 

o 

4-» 

CD 

_3 

5 O 
o "3 

= >* 
CD g 



CD 



CD 

_C 



t5 -x 

CD k 

CD 2 

CD Q) 



S'* 

o « 

-2 s 

E 

_ o> 

CD "O 
"O 3 



43 



CO o 



CO 



SERVE Assessment Training Questionnaire - Revised - 2002 



Appendix C 



ASSESSMENT QUESTIONAIRRE (Baseline Instrument) 



On the items below a twofold assessment is called for. The first part is the desirability index. 
You are rating you opinion about importance of the belief or practice using this scale. The 
second rating is an implementation index. To what extent do the teachers in your school 
implement the belief or practice? The rating scales are given below. For each item, please circle 
a rating from 1 to 5 in the desirability box and 1 to 5 in the implementation box to the right of 
the item. 



DESIRABILITY 


IMPLEMENTATION 


5 - High Desirability, of treat interest, highly valued 


5 - Excellent implementation 


4 - Above average desirability or value 


4 - Above average, good implementation 


3 - Average desirability or value 


3 - Average implementation 


2 - Of minimal importance or value 


2 - Weak implementation 


1 - Undesirable, of no value 


1 - No implementation 



Item 



Implementation 






Low 




Hi 


gh 


1. Conscious efforts are made to assess 












student progress relative to instructional goals 
that go beyond recall of information 


1 


2 


3 


4 


5 


2. Students' instructional needs are often 
assessed and instructional decisions are made 
based on student needs. 


1 


2 


3 


4 


5 


3. Students can articulately discuss their 
progress on key instructional goals. 


1 


2 


3 


4 


5 


4. Students are involved in self-evaluating 
their progress by gathering evidence (using a 
portfolio or other means) to show that they 
have mastered key competencies. 


1 


2 


3 


4 


5 


5. Teachers limit the number of assignments 
for which letter grades or percent cores are 
given so that students focus more on learning 
than worrying about "the grade" 


1 


2 


3 


4 


5 


6. Students often receive individualized 
feedback on assignments (oral and/or written 
comments) that will help them improve. 


1 


2 


3 


4 


5 


7. At this school, we never limit the number of 
high grades given in the class. 


1 


2 


3 


4 


5 


8. Students have many opportunities to make 
choice and take charge of the way they will 
learn a particular topic. 


1 


2 


3 


4 


5 


9. Teachers often talk with students about 
strengths and goals for improvement. 


1 


2 


3 


4 


5 



Low 
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Appendix C 



ASSESSMENT QUESTIONAIRRE (Baseline Instrument) 



Final Questions 

1. How do you assign grades for a reporting period? Briefly describe how you arrive at a report card grade and what 
percent of the grade reflects short answer, multiple-choice based tests and quizzes (assessment of basic, right/wrong. 
Factual knowledge). If grades do not apply (e.g. K-2), describe how you report achievement. 



2. Assume you are a teacher who uses a variety of alternative assessment strategies (such as those listed on the first page) 
in addition to traditional tests. What could you tell a teacher who uses primarily traditional (multiple-choice, short answer, 
matching) methods about how to use of these methods has impacted you or your students? 



3. We are interested in specific examples of non-traditional assessments used by teachers in your school. Think of an 
assessment (task, project, journal, portfolio, presentation, etc.) event that you think either had a positive impact on you or 
your students and describe below why you used it and why you liked it. 



Describe the assessment event: 



The purpose (why you used it and what it assessed): 



Why you liked it: 



4. Do you have any thought about where you would like to your school go in terms of assessment beliefs, practices, or 
skills? 
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